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Abstract 

Background: In the last few years there seems to be an emerging interest for including the patients' perspective 
in assessing methadone maintenance treatment (MMT), with treatment satisfaction surveys being the most 
commonly-used method of incorporating this point of view. The present study considers the perspective of 
patients on MMT when assessing the outcomes of this treatment, acknowledging the validity of this approach as 
an indicator. The primary aim of this study is to evaluate the concordance between improvement assessment 
performed by two members of the clinical staff (a psychiatrist and a nurse) and assessment carried out by MMT 
patients themselves. 

Method: Patients (n = 110) and their respective psychiatrist (n = 5) and nurse (n = 1) completed a scale for 
assessing how the patient's condition had changed from the beginning of MMT, using the Patient Global 
Impression of Improvement scale (PGI-I) and the Clinical Global Impression of Improvement scale (CGI-I), 
respectively. 

Results: The global improvement assessed by patients showed weak concordance with the assessments made by 
nurses (Quadratic-weighted kappa = 0.13, p > 0.05) and by psychiatrists (Quadratic-weighted kappa = 0.19, p = 
0.0086), although in the latter, concordance was statistically significant. The percentage of improved patients was 
significantly higher in the case of the assessments made by patients, compared with those made by nurses (90.9% 
vs. 80%, Z-statistic = 2.10, p = 0.0354) and by psychiatrists (90.9% vs. 50%, Z-statistic = 6.48, p < 0.0001). 

Conclusions: MMT patients' perception of improvement shows low concordance with the clinical staff's 
perspective. Assessment of MMT effectiveness should also focus on patient's evaluation of the outcomes or 
changes achieved, thus including indicators based on the patient's experiences, provided that MMT aim is to be 
more patient centred and to cover different needs of patients themselves. 



Background 

Methadone maintenance treatment (MMT) should be 
considered as a specific psychopharmacological treat- 
ment of heroin dependence [1] at the same time as 
being an essential and fundamental element of harm 
reduction strategies [2]. Since its introduction in heroin 
dependence management, MMT has undergone a con- 
stant process of review and evaluation. During the past 
fifteen years there have been a substantial number of 
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systematic reviews aiming at methodically and rigorously 
summarising available scientific evidence on the efficacy 
of this treatment (e.g., [1-4]). In all these systematic 
reviews, efficacy and effectiveness of MMT have been 
evaluated almost exclusively using so-called hard indica- 
tors or criteria [5,6]: retention on the programme, absti- 
nence from non-prescription opioids, reduced morbidity 
and mortality and/or reduced crime rate, among other 
variables. However, assessment of MMT should include 
patient's subjective evaluation of the treatment process 
and the changes achieved, by means of indicators based 
on their experiences [7]. 

At a more global level, several authors have pointed 
out the importance of including patient's perspective 
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when assessing health technologies and health services 
[8,9], and the potential significance of patient's perspec- 
tive towards research and generation of knowledge [10]. 
It should also be noted that there is a growing interest 
in incorporating patient's perspective in the develop- 
ment of clinical practice guidelines [11,12]. In fact, some 
of the most recent systems for establishing the strength 
of the recommendations (e.g., the GRADE approach 
[13]) include patient's preferences as one of the determi- 
nant factors: the higher the variability (i.e. uncertainty) 
of patients' values and preferences, the higher the prob- 
ability of a weak recommendation. However, the major- 
ity of clinical practice guidelines do not include 
evidence on patient's preferences, particularly given the 
limited number of studies available on this issue [14]. 
Notable exceptions are the Allergic Rhinitis and Its 
Impact on Asthma guidelines (ARIA) [15] which include 
in their recommendations, suggestions such as "in many 
patients with strong preference for the oral versus the 
intranasal route of administration, an alternative choice 
may be reasonable" [15], p. 471. 

MMT patients' perspective has received scant atten- 
tion from clinicians and researchers [16,17]. Tradition- 
ally, patients' point of view has not been considered a 
relevant contribution to the design, implementation and 
evaluation of MMT. However, over the last few years 
there seems to be an emerging interest for including 
patients' perspectives in assessing MMT. This new fra- 
mework is probably related to the gradual, albeit timid 
tendency to modify the hierarchised doctor-patient rela- 
tionship based on the traditional medical model of dis- 
ease, which implies a biased and exclusive perception of 
drug users as non-competent persons [18,19]. 

To date, MMT satisfaction surveys have been the 
most commonly-used method of incorporating patient's 
perspective [7]. However, other attempts have consid- 
ered and explored variables such as beliefs and views 
about methadone [20], perceived adjustment of the 
methadone dose [21], assessment of the relational 
dynamic established with other stakeholders [22], degree 
of participation in decision making [21], indicators of 
perceived quality [23] and views on the development 
and improvement of MMT [24]. Assessment of MMT 
outcomes from patient's perspective (i.e., evaluation of 
global perceived improvement in comparison with the 
pre-MMT situation) has not received the attention it 
deserves. To our knowledge, only one study [25] has 
recently and explicitly addressed this issue but without 
taking into account clinician's point of view about the 
same outcome, nor the concordance between both 
perspectives. 

This study considers the perspective of patients on 
MMT when assessing the outcomes of this treatment, 
acknowledging its validity as a relevant indicator. The 



primary aim of this study is to evaluate the concordance 
between improvement assessment performed by two 
members of the clinical staff (psychiatrists and nurses) 
and assessment carried out by MMT patients them- 
selves. It also analyses whether perceived improvement 
by MMT patients is associated with their satisfaction 
with treatment and with their views on methadone as a 
medication for treating heroin dependence. 

Method 

Participants 

Units of analysis in this study were 110 cases whose 
information was provided by three sources: 110 patients 
on MMT and their respective nurses and psychiatrists. 
Participants were methadone-maintained, opioid-depen- 
dent patients who had received MMT at our centre for 
at least 3 months, and who had signed an informed con- 
sent form. Each MMT patient has a psychiatrist who is 
responsible for their assessment, diagnosis, prescription 
and treatment, and a nurse in charge of monitoring and 
assessing patient status, handling daily medical issues 
and dispensing methadone. A total of five psychiatrists 
and one nurse participated in the study, with an average 
of 22 patients (range 2-46) per psychiatrist. 

Instruments 

To assess how the patient's condition had changed from 
the beginning of MMT, patients and their respective 
psychiatrist and nurse completed a scale: the Patient 
Global Impression of Improvement scale (PGI-I) and 
the Clinical Global Impression of Improvement scale 
(CGI-I) [26], respectively. These scales assess the degree 
in which the patient has improved in comparison with 
the pre-MMT situation, and consist of a single item in a 
7-point Likert format (1 = very much improved to 7 = 
very much worse). An additional rating analysis devel- 
oped by Demyttenaere et al. [27] was also used: 
improved (points 1 and 2), stable (points 3 to 5), and 
worsened (points 6 and 7). Satisfaction with MMT was 
assessed with the Verona Service Satisfaction Scale for 
Methadone Treatment (VSSS-MT) [28]. This scale has 
27 items and consists of four factors: basic interventions, 
specific interventions, social worker skills, and psycholo- 
gist skills. Items are based on a 5-point Likert scale (1 = 
terrible, 2 = mostly dissatisfied, 3 = mixed, 4 = mostly 
satisfied, 5 = excellent). The ranges of clinical signifi- 
cance for the VSSS-MT scores are [29]: 1-2 (very dissa- 
tisfied), > 2-3 (slightly dissatisfied), > 3-4 (slightly 
satisfied), and > 4-5 (very satisfied). Patients' opinion of 
methadone as a medication to treat heroin dependence 
was explored with the following question [21]: 'Taking 
into account your overall experience, what is your 
impression about methadone as a medication for carry- 
ing out maintenance treatment of heroin dependence?'. 
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The same 5-point Likert scale used as a response format 
in the VSSS-MT (1 = terrible to 5 = excellent) is estab- 
lished for this question. 

Procedure 

The research project was approved by the Clinical 
Research Ethics Committee of the Hospital de la Santa 
Creu i Sant Pau (Barcelona, Spain). A research psycholo- 
gist invited eligible patients to participate in the study 
and administered the surveys, which were conducted 
without the presence of clinical staff. No compensation 
was offered for participating in the study. When the 
clinicians completed the CGI-I scale for each patient, 
they were blind to patient's score on the PGI-I. The sur- 
veys were conducted between January and March 2007. 

Data analysis 

Quadratic- weighted kappa coefficients (« w ) were used to 
express the concordance between the clinical staff s and 
patient's own assessment of improvement. Z tests for 
comparing two proportions were used to determine 
whether the percentage of improved patients was statis- 
tically different when comparing the assessments made 
by clinicians with those made by the patients them- 
selves. Kendall's tau-b coefficients (zb) were calculated 
to examine the association between improvement 
assessed by the patient and the VSSS-MT scores or the 
patient's opinion of methadone as a medication. All sta- 
tistical tests were two-tailed and considered significant if 
p < 0.05. Results for the quadratic -weighted Kappa coef- 
ficients and Kendall tau-b coefficients were assessed 
using the ranges for effect size interpretation recom- 
mended by Ferguson [30]. Statistical analyses were per- 
formed using Epidat 3.1 (Direccion Xeral de Saude 
Publica de la Conselleria de Sanidade de la Xunta de 
Galicia and Pan American Health Organization) and 
SPSS Statistics 17.0 (SPSS Inc., Chicago, IL). 

Results 

Acceptance and completion of the survey 

Although there were 120 patients on MMT at our cen- 
tre at the time the study was performed, four had been 
on treatment for less than three months and so they 
were not invited to take part. Of the 116 surveys pro- 
posed, 110 (94.8%) patients agreed to participate and 
answered the survey. However, a psychiatrist did not 
complete the corresponding CGI-I scale for two 
patients. There were no statistically significant differ- 
ences regarding the sociodemographic and clinical char- 
acteristics of the patients who refused to participate in 
the survey, compared with those who accepted. 



Characteristics of the participants and MMT 

The 110 patients who completed the survey ranged in 
age from 22 to 58 years, with a mean of 39.4 years 
(standard deviation [SD] = 6.5). Males accounted for 
73.6% of the sample. With regard to marital status, 
63.3% were single, 19.3% married or living with a part- 
ner, 13.8% separated or divorced and 3.7% widowed. 
Participants used or had used heroin intravenously 
(56.9%), by inhalation (18.3%) or by snorting (24.8%). 
The mean of patients' total MMT episodes was 1.6 (SD 
= 0.9, range = 1-6). Participants were taking a metha- 
done dose of (mean ± SD) 67.2 ± 39.8 mg/d (range = 5- 
180). A percentage of 43.5 of patients were taking < 60 
mg/d of methadone, 42.4% were taking 60-100 mg/d, 
and 14.1% were taking > 100 mg/d. 

Relationship between patient and clinical staff 
assessment of improvement 

Concordance between global improvement assessed by 
psychiatrists and nurse was moderate and statistically 
significant (k w = 0.40, 95% Confidence Interval [CI] = 
0.26-0.55, p < 0.0001). However, the global improvement 
assessed by patients showed low concordance with the 
assessments made by nurse (ft w = 0.13, 95% CI = -0.05- 
0.31, p > 0.05; Table 1) and psychiatrists (« w = 0.19, 
95% CI = 0.08-0.30, p = 0.0086; Table 2), although the 
latter value reached statistical significance. 

By recoding scores of PGI-I and CGI-I into three cate- 
gories (improved, stable and worsened) [27], the percen- 
tage of improved patients was significantly higher in the 
case of the patients' own assessments compared with 
those made by nurse (90.9% vs. 80%, 95% CI for the dif- 
ference between proportions = 0.01-0.21, Z-statistic = 
2.10, p = 0.0354) and psychiatrists (90.9% vs. 50%, 95% 
CI for the difference between proportions = 0.29-0.53, 
Z-statistic = 6.48, p < 0.0001). 

Relationship between patient-assessed improvement and 
their satisfaction with MMT 

The global score in the VSSS-MT was (mean ± SD) 3.8 
± 0.5, and VSSS-MT factor scores were as follows: 3.9 ± 
0.6 in basic interventions, 3.5 ± 0.7 in specific interven- 
tions, 3.5 ± 1.0 in social worker skills, and 3.9 ± 0.9 in 
psychologist skills. All these scores indicated 'slight 
satisfaction' according to the VSSS-MT ranges of signifi- 
cance [29]. No statistically significant association was 
found between perceived improvement by MMT 
patients and their satisfaction with treatment. With 
regard to overall VSSS-MT score and factors, the Ken- 
dall tau-b values (-0.12 < xb < 0.09) did not reach statis- 
tical significance (Table 3). 
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Table 1 Concordance between patient and nurse perceptions of improvement 

Patient Global Impression of Improvement (PGI-I) scale 

Very much Much Minimally No Minimally Much Very Total 
improved improved improved change worse worse much 

worse 



Very much 61 12 1 1 2 77 

improved 

Much 8 2 1 11 

improved 

Minimally 4 2 1 11 9 

improved 

Clinical Global Impression of No change 4 4 11 10 

Improvement (CGI-I) scale 

Minimally 1 1 

worse 

Much worse 1 1 

Very much 1 1 

worse 

Total 80 20 3 3 1 1 2 110 

Data expressed as frequencies. 



Relationship between patient-assessed improvement and 
patient's opinion of methadone as a medication 

Although most participants had an excellent (27.3%) or 
mostly satisfied (56.4%) opinion of methadone as a med- 
ication for treating opioid dependence, almost a sixth of 
the participants expressed an opinion that was mixed 
(10.9%), mostly dissatisfied (1.8%) or terrible (3.6%). A 
statistically significant but weak association was found 
between perceived improvement by patients on MMT 
and their opinions of methadone as a medication (xb = 
-0.18, p = 0.042, Table 3). In fact, this correlation lost 
its statistical significance when patient-perceived 
improvement was classified by just three categories 
(improved, stable and worsened) [27]: ib = -0.13, p > 
0.05. 



Discussion 

The primary objective of this study was to assess the 
concordance between the assessment of improvement 
evaluated by two members of clinical staff (a psychiatrist 
and a nurse) and the assessment performed by MMT 
patients themselves. The results reveal that global 
improvement assessed by patient showed low concor- 
dance with nurse- and psychiatrist-assessments. It has to 
be mentioned that although patient-psychiatrist concor- 
dance reached statistical significance, it did not meet the 
recommended minimum effect size [30]. This low con- 
cordance between clinical staff s and patient's perspec- 
tive stems from the fact that patients' assessments are 
significantly more frequently positive. This patient-clini- 
cian discrepancy regarding perceived improvement is 



Table 2 Concordance between patient and psychiatrist perceptions of improvement 

Patient Global Impression of Improvement (PGI-I) scale 

Very much Much Minimally No Minimally Much Very Total 
improved improved improved change worse worse much 

worse 



Very much 29 4 33 

improved 

Much 15 5 1 21 

improved 

Minimally 21 4 1 1 2 29 

improved 

Clinical Global Impression of No change 8 4 2 2 16 

Improvement (CGI-I) scale 

Minimally 4 1 5 

worse 

Much worse 1 2 3 

Very much 1 1 

worse 

Total 78 20 3 3 1 1 2 108 



Data expressed as frequencies. 
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Table 3 Kendall's tau-b correlations of PGI-I with VSSS- 
MT scores and opinion of methadone as a medication 



PGI-I 

VSSS-MT, overall -0.032 

VSSS-MT, Basic interventions -0.001 

VSSS-MT, Specific interventions -0.063 

VSSS-MT, Social worker skills -0.1 06 

VSSS-MT, Psychologist skills 0.088 

Opinion of methadone as a medication -0.178* 



VSSS-MT: Verona Service Satisfaction Scale for Methadone Treatment; PGI-I: 
Patient Global Impression of Improvement scale; *p < 0.05, two-tailed. The 
scores on the PGI-I scale are in descending order of improvement (1 = very 
much improved to 7 = very much worse) while both the VSSS-MT and the 
scores on the opinion on methadone as a medication are in increasing order 
of satisfaction (1 = terrible to 5 = excellent). 

consistent with the results of the study conducted by 
Pulford et al. [31] comparing client (new admissions to 
an alcohol and other drug counseling community-based 
service) and clinician perspectives on problem improve- 
ment at two months of follow-up. Similarly, the main 
results of this study showed that clinician ratings of cli- 
ent improvement were significantly lower than ratings 
the clients gave themselves [31]. This patient-clinician 
discrepancy regarding patient improvement is also quite 
common, although not unanimous, in the field of men- 
tal health (see [32] for a review). 

A first possible explanation for this patient-clinician 
discrepancy regarding the assessment of MMT results 
could lie in the fact that patients may have a tendency 
to assess their progress more positively. This tendency 
would in turn be explained by one of the following facts 
or possibilities, which are not mutually exclusive: a) 
patients have a more detailed knowledge of their situa- 
tion before starting MMT and of their current condition 
and changes in relation to the previous situation, 
b) some clinicians may be too strict or demanding 
when allocating certain scores, c) some patients aware 
of their limited or lack of improvement, could overrate 
the improvement in order to reduce their cognitive 
dissonance. 

An alternative explanation for the patient-clinician 
discrepancy regarding the assessment of MMT results 
could stem from the fact that clinicians and patients 
emphasise different areas and outcomes when they 
define success, progress or improvement. This definition 
would largely depend on how MMT is conceptualised 
and what goals have been set for this treatment. As 
Koester et al. [33] showed, there are many different 
goals or reasons for a heroin user to enter and/or stay 
on MMT and they may differ from those of the profes- 
sionals who provide these treatments. In turn, these pro- 
fessionals also have a wide range of beliefs and attitudes 
about the goal of MMT [34] . However, all too often the 
goal of MMT is not negotiated and agreed with patient 



[35,36], implicitly and naively assuming that both the 
goal and, consequently, the definition of improvement 
are the same for the patient and clinician. Despite this 
assumption, a considerable number of authors (e.g., 
[7,33,37-39]) point out the need to consider patients' 
priorities when a) rethinking what is understood by a 
successful MMT or intervention, and b) establishing 
areas and outcomes that need to be assessed when 
establishing the effectiveness of these interventions. In 
fact, within some harm reduction services or programs, 
models and instruments to assess outcomes have been 
developed that are consistent with the fact that both the 
concepts of success and progress and their assessment 
should be reformulated in order to reflect the patient's 
perspective (e.g., [40,41]). 

Another notable finding of the present study is that 
MMT patients' perceived improvement is not associated 
with their satisfaction with MMT or with their views on 
methadone as a medication. These findings are consis- 
tent with previous results on a sample of patients on 
MMT by Perreault et al. [25] and by Ries et al. [42] in a 
study based on a sample of dually diagnosed outpatients 
participating in a long-term integrated dual disorder 
treatment. Correlational analyses of Perreault et al. [25] 
between several measures of perceived improvement 
and satisfaction with MMT revealed both non-signifi- 
cant associations and some statistically significant but 
weak correlations. All in all, these results seem to show 
that satisfaction with MMT is weakly related to the 
treatment outcome when assessed from patient's per- 
spective. Therefore, it could be suggested that these two 
variables, though slightly related, represent distinct 
approaches to MMT patient's perspective. In this regard, 
a study by Rademakers et al [43] found that, when 
exploring the extent to which satisfaction aspects deter- 
mine patients' overall satisfaction rating, aspects related 
to process were the most important predictors of this 
global rating, followed by aspects of structure and, lastly, 
aspects of outcome. 

This study is not without limitations. Its cross-sec- 
tional design does not permit causal relationships to be 
established between the different variables related to the 
patient's perspective. Although the sampling was 
exhaustive, the fact that patients and clinicians were 
from a single MMT centre, limits the generalisability of 
these findings to other MMT programs. Since both the 
PGI-I scale and the CGI-I scale are generic single-item 
tools, the detail of information obtained from these 
scales is somewhat limited. Moreover, due to the fact 
that both scales do not specify criteria for assessing the 
change experienced by patients with regard to their con- 
dition before starting MMT, it cannot be assured 
whether patients and clinicians have been guided by the 
same outcome areas and goals when assessing this 



Trujols et al. Harm Reduction Journal 201 1, 8:23 
http://www.harmreductionjournal.eom/content/8/1/23 



Page 6 of 7 



change. However in the case of the PGI-I scale, this very 
fact may have increased the relevance, significance or 
content validity of a scale developed by clinicians with- 
out the participation of patients themselves. Despite 
these limitations, the fact that parallel versions of the 
same scale (i.e., PGI-I and CGI-I) were used in this 
study to assess MMT patients' improvement, reinforces 
the robustness of the low concordance between patients 
and clinical staff when rating perceived improvement of 
MMT patients, found in this study. 

Further research is needed not only to assess the gen- 
eralisability of these findings to patients and clinicians 
from other MMT programs and/or geographic areas, 
but also to provide new data about patient-clinician 
concordance regarding MMT outcomes by including 
additional instruments for a deeper assessment of the 
matter. Moreover, future studies should also provide 
evidences, through qualitative research methods such as 
focal groups, in-depth interviews or cognitive interviews, 
about the outcome domains and/or variables that both 
patients and clinicians consider in assessing patient's 
improvement. 

Conclusions 

MMT patients' perception of improvement shows low 
concordance with the clinical staff s perspective. Assess- 
ment of MMT efficacy and effectiveness should also 
take into account patient's evaluation of the outcomes 
or changes achieved. Therefore, indicators based on the 
patient's experiences should be included, provided that 
MMT aim is to be more patient centred and to cover 
different needs of patients themselves. 
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